Distributed data handling and processing resources system

ABSTRACT

The distributed data handling and processing resources system of the present invention includes a) a number of data handling and processing resource nodes that collectively perform a desired data handling and processing function, each data handling and processing resource node for providing a data handling/processing subfunction; and, b) a low latency, shared bandwidth databus for interconnecting the data handling and processing resource nodes. In the least, among the data handling and processing resource nodes, is a processing unit (PU) node for providing a control and data handling/processing subfunction; and, an input/output (I/O) node for providing a data handling/processing subfunction for data collection/distribution to an external environment. The present invention preferably uses the IEEE- 1394 b databus due to its unique and specialized low latency, shared bandwidth characteristics.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to the partitioning and packaging of the elementsof a data handling and processing system and, more particularly, to adata handling and processing system using distributed data handling andprocessing resource elements and an interconnecting low latency network.

2. Description of the Related Art

Data handling and processing systems are called upon to form the heartof most of today's advanced technological products. Often theircomponents may be grouped together into convenient locations where aminimum mechanical and thermal environmental stress may be presented tothe components. In such cases, data handling and processing resourcesmay be functionally and physically partitioned into modules and housedtogether in multiple-module racks. Modules requiring communication withone another typically rely on a chassis backplane to effect suchcommunications over the relatively short distances of several inchessupported by the chassis backplane.

The increasing demands on processing throughput placed on today'ssystems require these systems to run at ever increasing speeds. Theseincreasing speeds translate into increased power consumption for suchmodules. The modules' collection together into a common chassis leads tohigher power densities for the chassis with modules, and increasingdemands on the amount of heat required to be removed from the chassisassembly by its associated thermal designs. Often times, the thermaldesign includes aggressive active systems to remove such heat, e.g., viathe use of forced air convection cooling, conduction cooling through themodules to the chassis and on to the chassis' baseplate, and coldplatemounting surfaces which may, in turn, have be actively cooled by liquidcooling loops. The active cooling required for most modern systems canbecome a particularly costly environmental support feature for someapplications, such as for space applications. Every pound of weightrequired to create such active cooling accommodations for on-boardelectronic processing systems can typically cost $10,000 to carry toorbit, at today's launch costs. Methods that could reduce or eliminatesuch active cooling requirements via, perhaps passive cooling methods,would be especially valuable for space systems. One method which couldhelp alleviate the active cooling requirement, replacing it with a formof passive cooling, would be to divide up processing resources which arenormally housed together into multiple-module chassis, packaging themseparately and providing an alternative means to communicate with oneanother than over the usual chassis backplane. With modules individuallypackaged, the temperature differentials required to remove heat from amodule could be reduced by reducing the length of the thermal path to anappropriate heat sink. The communication method used to interconnectmodules as a substitute for the backplane of the former chassis wouldhave to offer data transfer bandwidth, addressing flexibility, and timetransfer latency comparable to that of a backplane. Further, it would bedesirable to create a scalable, larger combined virtual processingcapability to accommodate the total processing needs of larger systems.

Some software methods have been developed for partitioning suchfunctions across an arbitrary network and connecting them together witha communications protocol. One such method and standard is known as theScalable Coherent Interface, and is defined by the open system standardIEEE-1596.

SUMMARY

In a broad aspect, the distributed data handling and processingresources system of the present invention includes a) a number of datahandling and processing resource nodes that collectively perform adesired data handling and processing function, each data handling andprocessing resource node for providing a data handling/processingsubfunction; and, b) a low latency, shared bandwidth databus forinterconnecting the data handling and processing resource nodes. In theleast, among the data handling and processing resource nodes, is aprocessing unit (PU) node for providing a control and datahandling/processing subfunction; and, an input/output (I/O) node forproviding a data handling/processing subfunction for datacollection/distribution to an external environment. The presentinvention preferably uses the IEEE-1394b databus due to its unique andspecialized low latency, shared bandwidth characteristics.

The present invention allows modules to be packaged separately foreasier heat removal, facilitating achievement of a passive coolingobjective. Those same modules may be located at more installationconvenient or environmentally permissive locations anywhere served bythe interconnecting databus. The use of a low latency databus in thepresent invention is a key enabling technology that allows theelimination of the conventional chassis backplane for intermodulecommunication.

The present invention is to be differentiated from the practice called“distributed computing.” With distributed computing, a typically massivedata processing task is partitioned into smaller pieces and thenparceled out to a collection of processing nodes, each connected to acentral coordinating node. Each processing node is a smaller, butcomplete, computing system of its own, consisting of modules usuallycommunicating with one another across the backplane of their commonchassis. An interconnecting databus network is used to communicate thetask pieces, including input data and output results between individualnodes and the central coordinating node.

The present invention is also to be distinguished from other distributedprocessing and data handling resource concept implementations thatutilize reflective memory to effect low latency communication betweenprocessing element nodes. Reflective memory mechanizations maintain areplicated shared memory space in each on the nodes that requireinternode communication to one another. Any change written into theshared reflective memory portion in one node is passed on to all othernodes and therein updated. Such shared memory blocks are a relativelysmall (e.g., 4 MByte) portion of the usually addressable address spacewithin each node. When larger address spaces must be transferred betweennodes, as is commonly the case, the reflective memory serves in thecapacity of an input/output buffer with the buffer data being rewrittento other memory locations within each node. One such reflective memorysystem is marketed under the brand name SCRAMNet® (see “Using SCRAMNet®for Cluster Computing: Early experiences,” by Matt Jancunski, et. al.,available on the Internet athttp://nowlab.cis.ohio-state.edu/projects/Papers/sc98.htm). The presentinvention does not rely on memory replication for data transfer, butinstead may directly address the memory space: within other nodes andtransport that data directly between large address spaces within eachnode. Such capability is provided by an appropriate addressing formatwithin the protocol of the databus. Additionally, the databusaccomplishes such transfers with a time latency comparable to thatachievable over conventional chassis backplanes. The present inventionis also to be distinguished from a similar technology and standard whichis known as the Scalable Coherent Interface, or SCI (see (1) “ScalableCoherent Interface (SCI),” available on the Internet at

http://www.oslo.sintef.no/ecy/projects/SCI_Europe/sci.html and (2)“SCI-Connected Clusters at the LfBS”, available on the Internet athttp://www.lfbs.rwth-aachen.de/users/joachim/SCI). As defined in thefirst page of the “Scalable Coherent Interface” reference, “SCI is anIEEE-standard which defines a protocol for remote memory-access ofdistributed compute systems. This kind of interconnect can be sortedunder the current term System Area Net (SAN)”. Its methods are definedby the IEEE-1596 standard. It presents a software and hardware basedmethodology for memory access between nodes, allowing them to operatecooperatively with one another as though they were a larger virtualprocessing machine. In this aspect, the present invention provides thesame capability, except that the computer nodes themselves need not becomplete computers themselves. The present invention does not dependupon the use of the IEEE-1596 SCI protocol elements but may, for theefficiency afforded by the use of such prior art, employ a subset ofthem in the buildup of the software drivers of the present invention.Further, as stated on page 2 of the reference “SCI-Connected Clusters atthe LfBS”, “The communication protocol is implemented in hardware andthus operates very efficiently”, indicating that the SCI is implementedwith specific hardware support to achieve its wide addressability, lowlatency data passing. The present invention, as will be disclosed inmore detail below, does not require such dedicated embedded hardwarebeyond that normally required for the interface to the databus itself.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic illustration of a preferred embodiment of thedistributed data handling and processing resources system of the presentinvention.

DETAILED DESCRIPTION OF THE INVENTION

Referring to the drawings and the characters of reference markedthereon, FIG. 1 illustrates a preferred embodiment of the distributeddata handling and processing resources system of the present invention,designated generally as 10. The system 10 includes a plurality of datahandling and processing resource nodes 12, 12′, 12″, . . . 12 ^(n) thatcollectively perform a desired data handling and processing function. Alow latency, shared bandwidth databus 14 interconnects the data handlingand processing resource nodes 12.

Each data handling and processing resource node 12 provides a desireddata handling/processing subfunction. As used herein a “datahandling/processing subfunction” means the performing coordination andcontrolling the flow of data through the system including, if necessary,conducting processing upon such data for the purpose of generating newdesired outputs. In the minimum, the system 10 must include a processingunit (PU) node 12 for providing a control and data handling andprocessing subfunction; and, an input/output (I/O) node 12′ forproviding a data handling/processing subfunction for datacollection/distribution to an external environment.

A control and data handling/processing subfunction may include, forexample, the processing normally provided in a conventional computer bya CPU. The PU node 12 may comprise a variable power microprocessor suchas those currently used in some space applications; a low powermicroprocessor such as IBM's new silicon-on-insulator processor, Intel'sTualatin™, or Advanced Micro Devices' Athlon™ 4; or, a microprocessorthat uses voltage and clock frequency scaling to achieve variable powercharacteristics, such as the Transmeta Corporation's Crusoe™ processor.

A data handling/processing subfunction for an input/output of node 12′for data collection/distribution may include, for example, remoteinterface units.

The low latency, shared bandwidth databus 14 preferably comprises anIEEE-1394b databus. The selection of IEEE-1394b with its efficient andprioritizable BOSS (Bus Owner/Supervisor/Selector) bus access protocoluniquely (relative to other bus protocol choices) provides thecapability of providing data transfer latencies comparable to that of abackplane. Its improved efficiency is also notable and extremelyimportant to achieving low latency by virtually eliminating thearbitration subaction gaps (timeout periods) previously used fornegotiating access to the bus in earlier (1394a) versions of the bus.Such performance further allows utilizing software elements of ascalable coherent interface standard, as necessary or desirable withoutother dedicated hardware chip support as conventionally required with anSCI interface. The scalable coherent interface standard may comprise theScalable Coherent Interface Standard IEEE-1596. The low latency, sharedbandwidth databus may be implemented, for example, as a dual-redundantdatabus, comprising two IEEE-1394b databuses. It could also beimplemented as a dual-redundant databus, comprising a single IEEE-1394bdatabus configured as a dynamically reconfigurable loop to achievefail-and-still-operate capability. The low latency, shared bandwidthdatabus may provide low message packet sizes to achieve low latency.

As used herein, the term “low” in referring to the latency means alatency of less then 100 microseconds. The databus preferably has alatency of between about 0.5 to 10 microseconds.

One of the data handling and processing resource nodes 12 preferablyincludes a memory node 12″ for providing a memory subfunction. Thememory node 12″ may be volatile or non-volatile depending on the desiredapplication. This might include expanded memory for running largeapplication programs which do not have sufficient room to run in thememory contained totally within the processing element modules, 12 or12′″. Such memory modules 12″ might also contain non-volatile memorystorage for collecting and retaining on-board vehicle health orperformance data.

Other data handling and processing resource nodes 12 may comprisespecial purpose data handling and processing resource nodes such asmodules for providing subsystem control functions for vehicle enginevalves or servos for guidance control surfaces.

The system 10 of the present invention may include a passive coolinginterface assembly 16 for connection to a passive cooling system (notshown). The passive cooling interface assembly 16 may include, forexample, linear or loop heat pipes, with various internal phase changeliquid mechanims therein, or may simply be totally static thermalconduction paths fabricated from metallic cables or special thermalmaterials. The heat may be removed to a hardmount or, via copper or aheat pipe cable, removed to a more distant heat sink location. Thepassive cooling interface assembly 16 is operatively associated with thedata handling and processing resource nodes 12. It is interfaced to thenodes' enclosure for transferring heat away from the enclosure.

In this preferred embodiment, using the IEEE-1394b databus, the datahandling and processing resource nodes 12 receive their operating powerfrom power cables 18 that are commingled with the databus 14.

The data handling and processing resource nodes 12 may employ high powerconversion efficiencies from respective power sources associatedtherewith. Such power sources may be, for example, power carried withinthe signal cable harness assembly, or supplied separately to the unitvia separate power cable harness lines. The high power conversionefficiencies may be in a range of from about 85 to 95 percent.

Preferably, an application program interface (API) is interposed betweenthe low latency, shared bandwidth databus 14 and a user for segmentinglarge messages into smaller pieces. Such an API provides the function ofcreating gaps within larger messages so that other messages that requirelow latency bus access can gain timely access to the bus without havingto wait for total completion of large messages. The databus 14 mayinclude a large, directly addressable, non-overlapping address spaceserving all of the data handling and processing resource nodes 12. Theaddress space may be greater than 1 terabyte.

During the typical operation of the system of the present invention,data is usually gathered by sensors external to the system and providedto the system through legacy signal input/output modules 12′. Such datais typically relayed to a processing element module 12 or 12′″, wherecollected data is processed by software algorithms residing in theprocessing elements, creating a new desired output. Such output is thentypically sent to its destination external to the system, again viaanother input/output module 12′, for obtaining its desired effect on thevehicle. During the course of such processing, the processing elementmay depend on the use of the extended or mass memory module 12″ toassist the program execution, or to store some of the results innon-volatile memory for later use. In some cases, an end effector (e.g.,an engine valve or actuator servo) may require local support for controlloop closure or signal conditioning, which may be accomplished with theuse of a special purpose module 12″.

The present invention, in addition to facilitating the objectives ofpassive cooling and more convenient installation locations, also has theadvantage of reducing the total weight of the system required to performa given total data handling and processing function. This arises fromthe ability to discard the conventional backplane and chassis requiredin conventional modular avionics approaches.

The present invention is particularly adaptable for use with an avionicscore processing system. One principal application for the presentinvention is for space vehicle applications, where considerable weightsavings over the conventional chassis approach may be expected. Also,the use of low power techniques discussed herein will allow significantreductions in power for the total system. Lastly, the potential forachieving passively cooled avionics can result in a significant weightsavings associated with conventional cooling techniques used withconventional modular avionics systems. It may also be used for anavionics peripheral processing system, or as a data handling systemwithout conventional processing functions for the coordination of dataflows in communications streams.

Thus, while the preferred embodiments of the devices and methods havebeen described in reference to the environment in which they weredeveloped, they are merely illustrative of the principles of theinventions. Other embodiments and configurations may be devised withoutdeparting from the spirit of the inventions and the scope of theappended claims.

1-39. (canceled)
 40. A distributed data handling and processing systemcomprising: a plurality of heat sink elements; a plurality of resourcenodes collectively configured to perform a desired data handling andprocessing function, each of said resource nodes configured to performone desired data handling and processing sub-function, each of saidresource nodes remotely packaged such that a thermal path to one of saidheat sink elements is minimized; and a low latency, shared bandwidthdatabus for interconnecting said plurality of resource nodes.
 41. Adistributed data handling and processing system according to claim 40wherein said plurality of resource nodes comprises at least oneprocessing unit node configured for control, data handling, and dataprocessing sub-functions.
 42. A distributed data handling and processingsystem according to claim 40 wherein said plurality of resource nodescomprises at least one input/output node configured for data collectionand distribution with respect to an external environment.
 43. Adistributed data handling and processing system according to claim 40wherein said plurality of resource nodes comprises at least one memorynode configured to provide a memory subfunction.
 44. A distributed datahandling and processing system according to claim 40 wherein said lowlatency, shared bandwidth databus comprises a IEEE-1394b databus.
 45. Adistributed data handling and processing system according to claim 40further comprising an application programming interface (API) configuredto segment messages from said databus into multiple, smaller messagesfor communication with an environment external to said distributed datahandling and processing system.
 46. A distributed data handling andprocessing system according to claim 45 wherein said API is configuredto create gaps within the messages so that other messages can havetimely access to said databus.
 47. A distributed data handling andprocessing system according to claim 40 wherein said low latency, sharedbandwidth databus comprises a directly addressable, non-overlappingaddress space serving all of said resource nodes.
 48. A method forreducing a total weight of a system configured to perform a given datahandling and processing function, said method comprising: dividing thesystem into a plurality of resource nodes collectively configured toperform the data handling and processing function, each of the resourcenodes configured to perform at least one sub-function of the desireddata handling and processing function; providing a plurality of heatsink elements; remotely packaging each of the resource nodes such that athermal path to one of the heat sink elements is minimized for eachresource node; and interconnecting the plurality of resource nodes witha low latency, shared bandwidth databus.
 49. A method according to claim48 wherein dividing the system into a plurality of resource nodescomprises: configuring at least one of the resource nodes as aprocessing node; and configuring at least one of the resource nodes asan input/output node configured for data collection and distributionwith respect to an external environment.
 50. A method according to claim49 wherein dividing the system into a plurality of resource nodescomprises configuring at least one of the resource nodes as a memorynode configured to provide a memory subfunction.
 51. A method accordingto claim 48 wherein interconnecting the plurality of resource nodescomprises interconnecting the plurality of resource nodes utilizing aIEEE-1394b databus.
 52. A method according to claim 48 furthercomprising configuring the databus to include a directly addressable,non-overlapping address space serving all of the resource nodes.
 53. Amethod according to claim 48 further comprising segmenting messages fromthe databus into multiple, smaller messages for communication with anenvironment external to the system
 54. A method according to claim 53wherein segmenting messages comprises creating gaps within the messagesto provide timely access to the databus for other messages.
 55. Adistributed data handling and processing system comprising: a pluralityof heat sink elements; at least one processing unit resource nodeconfigured for control, data handling, and data processingsub-functions; at least one input/output resource node configured fordata collection and distribution with respect to an externalenvironment; at least one special purpose resource node configured toperform a subsystem control function, each of said resource nodesremotely and separately packaged such that a thermal path to one of saidheat sink elements is minimized; and a databus for interconnecting saidresource nodes, said databus incorporating message packet sizes tocontrol a latency of messages between said resource nodes.
 56. Adistributed data handling and processing system according to claim 55wherein said databus is configured such that messages between saidresource nodes have a latency between about 0.5 and 10 microseconds. 57.A distributed data handling and processing system according to claim 55wherein said databus comprises a IEEE-1394b databus.
 58. A distributeddata handling and processing system according to claim 55 furthercomprising a passive cooling interface assembly operable for connectionto a passive cooling system, said resource nodes separately connectableto said passive cooling interface assembly.
 59. A distributed datahandling and processing system according to claim 55 wherein each saidresource node comprises a separate enclosure, said separate enclosuresindependently connectable to one of said heat sink elements.